<!DOCTYPE html>
<html lang="en">
<head>
        <title>Stratix V accelerator card from eBay, part 2</title>
        <meta charset="utf-8" />
	<meta name="viewport" content="width=device-width, initial-scale=1.0" />
        <link rel="shortcut icon" href="./theme/images/favicon.ico"/>
        <link rel="stylesheet" href="./theme/css/main.css" type="text/css" />
        <link href="www.j-marjanovic.io/feeds/all.atom.xml" type="application/atom+xml" rel="alternate" title="j-marjanovic.io Atom Feed" />

        <!--[if IE]>
                <script src="http://html5shiv.googlecode.com/svn/trunk/html5.js"></script><![endif]-->

        <!--[if lte IE 7]>
                <link rel="stylesheet" type="text/css" media="all" href="./css/ie.css"/>
                <script src="./js/IE8.js" type="text/javascript"></script><![endif]-->

        <!--[if lt IE 7]>
                <link rel="stylesheet" type="text/css" media="all" href="./css/ie6.css"/><![endif]-->
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.8/jquery.min.js" type="text/javascript"></script>
<script type="text/javascript" src="theme/jquery.cookiesdirective.js"></script>

</head>

<body id="index" class="home">
<script type="text/javascript">
	// Using $(document).ready never hurts
	$(document).ready(function(){

		// Cookie setting script wrapper
		var cookieScripts = function () {
			// Internal javascript called
			console.log("Running");
		
			// Loading external javascript file
			$.cookiesDirective.loadScript({
				uri:'external.js',
				appendTo: 'eantics'
			});
		}
	
		/* Call cookiesDirective, overriding any default params
		
			*** These are the defaults ***
				explicitConsent: true,
				position: 'top',
				duration: 10,
				limit: 0,
				message: null,				
				cookieScripts: null,
				privacyPolicyUri: 'privacy.html',
				scriptWrapper: function(){},	
				fontFamily: 'helvetica',
				fontColor: '#FFFFFF',
				fontSize: '13px',
				backgroundColor: '#000000',
				backgroundOpacity: '80',
				linkColor: '#CA0000'
				
		*/
		
		$.cookiesDirective({
			privacyPolicyUri: 'myprivacypolicy.html',
			explicitConsent: false,
			position : 'bottom',
			scriptWrapper: cookieScripts, 
			cookieScripts: 'Google Analytics', 
			backgroundColor: '#52B54A',
			linkColor: '#ffffff'
		});
	});
</script>

	
  <!-- <header id="banner" class="body"> -->
  <!--               <h1><a href="./"><img src="http://www.launchyard.com/images/logo.png" /></a></h1> -->
  <!--       </header> --> 

  <div class="LaunchyardDetail" style="align:right;">
    <!-- <p> -->
    <!-- <img src="./theme/images/blue-pin.png" width="100" height="100" alt="Graph icon"> -->
    <!-- </p> -->
    <p><a id="sitesubtitle" href="./">j-marjanovic.io</a></p>

	<br>
    <p style="float: right; margin-right: 50px;"><a id="aboutlink" href="./pages/about.html">About</a></p>

    <br>
	<p style="float: right; margin-right: 50px;"><img src="./theme/images/icons/rss.png"> <a id="aboutlink" href="./feeds/jan-marjanovic.atom.xml">Atom feed</a></p>
    <br>

  </div>

<section id="content" >
    <div class="body">
      <article>
        <header>
          <h1 class="entry-title">
            <a href="./stratix-v-accelerator-card-from-ebay-part-2.html" rel="bookmark"
               title="Permalink to Stratix V accelerator card from eBay, part 2">Stratix V accelerator card from eBay, part 2</a></h1>

        </header>

        <div class="entry-content">
<div class="post-info">
	<ul>
        <li class="vcard author">
                 by&nbsp;<a class="url fn" href="./author/jan-marjanovic.html">Jan Marjanovic</a>
        </li>
        <li class="published" title="2020-06-07T16:00:00+02:00">
          on&nbsp;Sun 07 June 2020
        </li>

	</ul>

</div><!-- /.post-info -->          <p>In my previous blog post I have explored the FPGA card I have purchased on eBay,
and in this post I will present an adapter card which I have developed. The
adapter provides PCI Express connection between a normal card-edge slot and the
FPGA card, as well as the access to the I2C bus and some additional signals.</p>
<p>To summarize my previous blog post, the FPGA card has a 160-pin Samtec
connector, providing power (12V), I2C bus for management, and a total of 16
lanes for PCI Express.</p>
<h1>The adapter</h1>
<p>To keep the cost down, I have tried to make the adapter as small as possible,
and at this point to only develop a proof-of-concept. On this first variant
of the adapter only connects one PCIe lane, and I plan a second variant where
all 8 or 16 lanes will be connected to the PCIe edge connector.</p>
<p>The KiCad project for the adapter card is available on <a href="https://github.com/j-marjanovic/ocs-tray-mezzanine-adapter">my
GitHub</a>.</p>
<p>Shown in the image below are the relevant parts of the adapter card.</p>
<p><img alt="Block diagram of the adapter card" src="./images/2020_fpga_card_part_2/otma.png" style="width:80%; display: block; margin-left: auto; margin-right: auto;"></p>
<p>To connect to the PCIe slot I have purchased one of those <a href="https://lmgtfy.com/?q=bitcoin+pci+express+riser+card&amp;t=i">Bitcoin mining riser
cards</a>, which provide
connections between PCIe slot and a USB3 cable. Since there are a total of
4 PCIe reference clock inputs per OCS specification, I have distributed
the clock to all inputs using a dedicated IC. I have connected the management
I2C bus (<code>MEZZ_SDA</code> and <code>MEZZ_SCL</code>) to a header, which allows me to explore the
bus with a Raspberry Pi for example, as shown in the picture below.</p>
<p><img alt="Raspberry Pi connected to the management I2C bus" src="./images/2020_fpga_card_part_2/adapter_rpi.jpg" style="width:80%; display: block; margin-left: auto; margin-right: auto;"></p>
<h1>I2C</h1>
<p>Having a Raspberry Pi connected to the I2C bus, we can first explore the
present devices.</p>
<p>Using <code>i2cdetect</code> we can find all devices which have acknowledged their I2C
address:</p>
<div class="highlight"><pre><span></span><code>pi@raspberrypi:~ $ i2cdetect -y 1
     0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f
00:          -- -- -- -- -- -- -- -- -- -- -- -- -- 
10: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
20: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
30: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
40: -- -- -- -- -- -- -- -- -- -- -- -- 4c -- -- -- 
50: -- 51 -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
60: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
70: -- -- -- -- -- -- -- 77         
</code></pre></div>


<p>We see the EEPROM at the address 0x51, and another two devices at addresses 0x4C
and 0x77 - I would assume that these are some kind of sensors and/or regulators.</p>
<h2>IPMI FRU</h2>
<p>We can now use the <code>i2cdump</code> to dump the content of the EEPROM. As I have
expected, the EEPROM is used by the Baseboard Management Controller (or
something similar) and the content is compliant with <a href="https://www.intel.com/content/dam/www/public/us/en/documents/product-briefs/platform-management-fru-document-rev-1-2-feb-2013.pdf">IPMI Platform Management
FRU Information Storage
Definition</a>.</p>
<p>The following information are stored in the EEPROM:</p>
<div class="highlight"><pre><span></span><code>ChassisArea:
  version: 1
  length: 32 bytes
  chassis type: Rack Mount Chassis
  part nr: X907370-001
  serial nr: (len 0)
  checksum: 146 (OK)
BoardArea:
  version: 1
  length: 64 bytes
  lang: 25
  mfg date: 2016-01-06 03:26
  mfgr: Microsoft
  prod_name: PPFPGA
  serial: OLJ60100194
  part: X900563-001
  file id: FRU 1.0
  checksum: 108 (OK)
ProductArea:
  version: 1
  length: 72 bytes
  mfgr: Microsoft
  prod_name: PPFPGA
  part_nr: X900563-001
  part_ver: 1.0
  part_sn: OLJ60100194
  asset_tag:
  file id: 1.0
  checksum: 241 (OK) 
</code></pre></div>


<p>From my understanding of the IPMI standard, the Chassis information should
not be present on this card:</p>
<blockquote>
<p>A system can have multiple FRU Information Devices within a chassis, but only
one device should provide the Chassis Info Area. Thus, this area will
typically be absent from most FRU Information Devices.</p>
</blockquote>
<h1>PCI Express</h1>
<p>And finally, the most interesting and the most challenging part, the PCI Express
connection.</p>
<p>I have plugged in the card, as shown on the image below and turned on the computer. </p>
<p><img alt="PCIe connection over USB cable" src="./images/2020_fpga_card_part_2/adapter_pcie.jpg" style="width:80%; display: block; margin-left: auto; margin-right: auto;"></p>
<p>And it works! Using <code>lspci</code> command to list all the devices visible to the CPU,
one can also note the Microsoft card:</p>
<div class="highlight"><pre><span></span><code>$ lspci
00:00.0 Host bridge: Intel Corporation 2nd Generation Core Processor Family DRAM Controller (rev 09)
00:01.0 PCI bridge: Intel Corporation Xeon E3-1200/2nd Generation Core Processor Family PCI Express Root Port (rev 09)
00:02.0 VGA compatible controller: Intel Corporation 2nd Generation Core Processor Family Integrated Graphics Controller (rev 09)
00:16.0 Communication controller: Intel Corporation 6 Series/C200 Series Chipset Family MEI Controller #1 (rev 04)
00:16.2 IDE interface: Intel Corporation 6 Series/C200 Series Chipset Family IDE-r Controller (rev 04)
00:16.3 Serial controller: Intel Corporation 6 Series/C200 Series Chipset Family KT Controller (rev 04)
00:19.0 Ethernet controller: Intel Corporation 82579LM Gigabit Network Connection (Lewisville) (rev 04)
00:1a.0 USB controller: Intel Corporation 6 Series/C200 Series Chipset Family USB Enhanced Host Controller #2 (rev 04)
00:1b.0 Audio device: Intel Corporation 6 Series/C200 Series Chipset Family High Definition Audio Controller (rev 04)
00:1c.0 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 1 (rev b4)
00:1c.4 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 5 (rev b4)
00:1c.6 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 7 (rev b4)
00:1d.0 USB controller: Intel Corporation 6 Series/C200 Series Chipset Family USB Enhanced Host Controller #1 (rev 04)
00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev a4)
00:1f.0 ISA bridge: Intel Corporation Q67 Express Chipset LPC Controller (rev 04)
00:1f.2 SATA controller: Intel Corporation 6 Series/C200 Series Chipset Family 6 port Desktop SATA AHCI Controller (rev 04)
00:1f.3 SMBus: Intel Corporation 6 Series/C200 Series Chipset Family SMBus Controller (rev 04)
03:00.0 Unassigned class [ff00]: Microsoft Corporation Device b100 (rev 01)
04:00.0 USB controller: NEC Corporation uPD720200 USB 3.0 Host Controller (rev 04)
05:03.0 FireWire (IEEE 1394): LSI Corporation FW322/323 [TrueFire] 1394a Controller (rev 70)
</code></pre></div>


<p>Using <code>lspci -vv</code> to get more information, we see that the link is established
at 2.5 GT/s (probably too many connectors in series to go faster) and at
x1 width (as expected, since we only pass one lane through the USB cable):</p>
<div class="highlight"><pre><span></span><code>$ sudo lspci -s 03:00 -vv
03:00.0 Unassigned class [ff00]: Microsoft Corporation Device b100 (rev 01)
    Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
    Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast &gt;TAbort- &lt;TAbort- &lt;MAbort- &gt;SERR- &lt;PERR- INTx-
    Latency: 0, Cache Line Size: 64 bytes
    Interrupt: pin A routed to IRQ 11
    Region 0: Memory at fa000000 (32-bit, non-prefetchable) [size=16M]
    Region 2: Memory at fb000000 (32-bit, non-prefetchable) [size=1K]
    Capabilities: [50] MSI: Enable- Count=1/4 Maskable- 64bit+
        Address: 0000000000000000  Data: 0000
    Capabilities: [78] Power Management version 3
        Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
        Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
    Capabilities: [80] Express (v2) Endpoint, MSI 00
        DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s &lt;64ns, L1 &lt;1us
            ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset- SlotPowerLimit 10.000W
        DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq-
            RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop+
            MaxPayload 128 bytes, MaxReadReq 128 bytes
        DevSta: CorrErr+ NonFatalErr- FatalErr- UnsupReq+ AuxPwr- TransPend-
        LnkCap: Port #1, Speed 8GT/s, Width x8, ASPM not supported
            ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
        LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk+
            ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
        LnkSta: Speed 2.5GT/s (downgraded), Width x1 (downgraded)
            TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
        DevCap2: Completion Timeout: Range ABCD, TimeoutDis+, NROPrPrP-, LTR-
             10BitTagComp-, 10BitTagReq-, OBFF Not Supported, ExtFmt-, EETLPPrefix-
             EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
             FRS-, TPHComp-, ExtTPHComp-
             AtomicOpsCap: 32bit- 64bit- 128bitCAS-
        DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
             AtomicOpsCtl: ReqEn-
        LnkCtl2: Target Link Speed: 8GT/s, EnterCompliance- SpeedDis-
             Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
             Compliance De-emphasis: -6dB
        LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete-, EqualizationPhase1-
             EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
    Capabilities: [100 v1] Virtual Channel
        Caps:   LPEVC=0 RefClk=100ns PATEntryBits=1
        Arb:    Fixed- WRR32- WRR64- WRR128-
        Ctrl:   ArbSelect=Fixed
        Status: InProgress-
        VC0:    Caps:   PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
            Arb:    Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256-
            Ctrl:   Enable+ ID=0 ArbSelect=Fixed TC/VC=01
            Status: NegoPending- InProgress-
    Capabilities: [200 v1] Vendor Specific Information: ID=0000 Rev=0 Len=044 &lt;?&gt;
    Capabilities: [300 v1] Secondary PCI Express
        LnkCtl3: LnkEquIntrruptEn-, PerformEqu-
        LaneErrStat: 0
    Capabilities: [800 v1] Advanced Error Reporting
        UESta:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
        UEMsk:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
        UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
        CESta:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr-
        CEMsk:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
        AERCap: First Error Pointer: 00, ECRCGenCap- ECRCGenEn- ECRCChkCap- ECRCChkEn-
            MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap-
        HeaderLog: 00000000 00000000 00000000 00000000
</code></pre></div>


<h1>Summary and outlook</h1>
<p>The adapter card did its job and provided access to the I2C bus and a connection
to the PCI Express. I have managed to parse the content of the EEPROM (and
realize that there is nothing interesting there) and establish the PCIe
connection to the FPGA, as a first step of getting the hardware ready for custom
developments.</p>
<p>Eventually I plan to develop a card with a wider PCIe link and get rid of the
USB cable setup.</p>
<p>There were some mistakes on the adapter board (rotated Samtec connector, swapped
TX and RX on the USB connector) which I could work around, and serve as a lesson
to be more careful next time and double-check everything.</p>
<p>As the next step, I would like to understand better how the JTAG chip works. I
know that it can be used with OpenOCD, but I would imagine that one can somehow
also make it talk to Quartus directly.</p>
<hr>
<p>All trademarks and registered trademarks are the property of their respective owners.</p>
        </div><!-- /.entry-content -->
<a href="https://twitter.com/share" class="twitter-share-button" data-count="horizontal" data-via="janmarjanovic">Tweet</a><script type="text/javascript" src="https://platform.twitter.com/widgets.js"></script><br/><br/>

      </article>
    </div>
</section>
        <section id="extras" >
        
        </section><!-- /#extras -->
	
        <footer id="contentinfo" >
                <address id="about" class="vcard ">
                Proudly powered by <a href="https://getpelican.com/" target="_blank">Pelican</a>, which takes
                great advantage of <a href="https://python.org" target="_blank">Python</a>.
		
                </address><!-- /#about -->
		

                
        </footer><!-- /#contentinfo -->

<script>
  (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
  (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
  m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
  })(window,document,'script','//www.google-analytics.com/analytics.js','ga');

  ga('create', 'UA-56555055-1', 'auto');
  ga('send', 'pageview');

</script></body>
</html>